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Abstract — In this paper, we account for approaches of sparse 
recovery from large underdetermined hnear models with per- 
turbation present in both the measurements and the dictio- 
nary matrix. Existing methods have high computation and 
low efficiency. The total least-squares (TLS) criterion has well- 
documented merits in solving linear regression problems while 
FOCal Underdetermined System Solver (FOCUSS) has low- 
computation complexity in sparse recovery. Based on TLS 
and FOCUSS methods, the present paper develops more fast 
and robust algorithms, TLS-FOCUSS and SD-FOCUSS. TLS- 
FOCUSS algorithm is not only near-optimum but also fast in 
solving TLS optimization problems under sparsity constraints, 
and thus fit for large scale computation. In order to reduce the 
complexity of algorithm further, another suboptimal algorithm 
named SD-FOCUSS is devised. SD-FOCUSS can be applied in 
MMV (multiple-measurement-vectors) TLS model, which fills 
the gap of solving linear regression problems under sparsity 
constraints. The convergence of TLS-FOCUSS algorithm and 
SD-FOCUSS algorithm is established with mathematical proof. 
The simulations illustrate the advantage of TLS-FOCUSS and 
SD-FOCUSS in accuracy and stability, compared with other 
algorithms. 

Index Terms — perturbation, linear regression model, sparse 
solution, optimal recovery, convergence, performance. 



I. Introduction 

The problem of finding sparse solutions to underdetermined 
system of linear equations has been a hot spot of researches in 
recent years, because of its widespread application in compres- 
sive sensing/sampling (CS)|[T], lHJ, biomagnetic imagining|3|, 
source localization ||4l, signal reconstruction ||6|, etc. 

In the noise-free setup, CS theory holds promise to explain 
the equivalence between ^g-norm minimization and ^i-norm 
minimization as solving exactly linear equations when the 
unknown vector is sparse|7|, |8|. Variants of CS for "noise 
setup" of perturbed measurements are usually solved based 
on basis pursuit (BP) approach|9l, lITOl (utilizing method 
of linear programming [4] or Lasso lfTTl ). greedy algorithms 
(e.g. OMPIIT2I. ROMPIIT3I. CoSaMPlIH, etc) or least-squares 
methods with ^i-regularization (e.g., FOCUSS|3], llM, B). 
However, exiting BP, greedy algorithms and FOCUSS do not 
account for perturbations present in the dictionary matrix, i.e. 
regression matrix. 

Recently, only a little attention has been paid on the sparse 
problems with perturbations present both in measurements and 
dictionary matrix. Performance analysis of CS and BP methods 
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for the linear regression model under sparsity constraints was 
researched in lfT6l . iflTJ and [18]; a feasible approach in |19| , 
named S-TLS, was devised to reconstruct sparse vectors based 
on Lasso from the "fully-perturbed" linear model. However, 
the research of 1 16l . ifTTl and flSl are limited in theoretical 
aspect and do not devise systematic approaches. Due to its 
highly-computational burden, S-TLS is very time-consuming, 
and thus unsuitable for large scale problems. 

In this paper, an extension form of FOCUSS is devised 
solving sparse problems to "fully-perturbed" linear model. 
Belonging to categories of convex optimization, LP and Lasso 
have the stable results but their computational burden is the 
highest; greedy algorithms have low computation, but their 
performances can only be guaranteed when the dictionary 
matrix satisfies some rigorous conditions, such as very small 
restricted isometry constants fl]. FOCUSS was originally 
designed to obtain a sparse solution by successively solving 
quadratic optimization problems and was widely used to deal 
with compressed sensing problems. The obvious advantages 
of FOCUSS are its low computation and stable results. For 
FOCUSS, only a few iterations tends to be enough to obtain a 
rather good approximating solution. So it is an excellent choice 
to develop FOCUSS to solve approximate sparse solutions to 
linear regression model, especially in large scale application. 

Our objective is to overcome the influence of perturbation 
present in dictionary matrix and measurements on the accu- 
racy of sparse recovery effectively. Meanwhile, the merits of 
FOCUSS, rapid convergence and good adaption to intrinsic 
properties of dictionary matrix, are maintained. First, objective 
function to be optimized can be obtained under a Bayesian 
framework. Then the necessary condition for the optimizing 
solution is that each first-order partial derivative of objective 
function is equal to zero. Next we can get the iterative 
expression using iterative relaxation algorithm. Finally, the 
new algorithms are proved to be convergent. 

The paper is organized as follows. In Section HI] we intro- 
duce perturbed linear regression model for sparse recovery, and 
analyze the optimal problem simply. In Section |lll] we use a 
MAP estimate to obtain the objective function to be optimized, 
then yield an iterative algorithm to provide solutions, named 
TLS-FOCUSS for adopting TLS method and framework of 
FOCUSS. Convergence of TLS-FOCUSS is proved. In Section 
IIVI we propose another algorithm based on FOCUSS and TLS 
model, named SD-FOCUSS to distinguish TLS-FOCUSS. 
Though SD-FOCUSS is a suboptimal optimal, its computation 
is low and it can be used in MMV case. In the simulation 
of Section |V] the performances of mentioned algorithms are 
presented. Finally, we draw some conclusions in Section [VTl 
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II. Perturbed Linear Regression Model 



III. TLS-FOCUSS Algorithm 



Consider the underdetermined linear system of y = Ax, 
where A is an m x n matrix with m < n, y is the given 
m X 1 data vector, and x is unknown n x 1 vector to 
be recovered. With x being sparse, and A satisfying some 
property (e.g., RIPfT|), CS theory asserts that exact recovery 
of X can be guaranteed by solving the convex problem|9|, |7|, 
II20I : min^; ||x||i s.t.(subject to) y = ^x. Suppose that data 
perturbations exist in the linear model Ax. The corresponding 
convex problem can be written as a Lagrangian formic, H, 
min, ||y - Ax\\l + j\\xP, where || • ||p = E I ' 1^ 7 > 
is a sparsity-tuning parameterfTP], and < p < 1 (p is set to 
1 in [91, HH). What the present paper focusses on is how to 
reconstruct sparse vector efficiently from over- and especially 
under-determined linear regression models while perturbations 
are present in y and/or A. 

The perturbed Unear regression model can be formulated as 
followslEfl, II2I : 



y = {A + E)x + e, 



(1) 



where e represents perturbation vector and E represents pertur- 
bation matrix. Due to randomness and uncertainty, it is usually 
assumed that the components of noise in the same channel 
are independently and identically Gaussian distributed, e.g. 
e ~ N{0,all) and vec(£;) 7V(0,cr|/), where vec(-) is 
matrix vectorizing operator. 
([T]) can be rewritten as 



{B + D) 



= 0, 



where B = [—y,A], D = [e,£']. Without exploiting spar- 
sity, TLS has well documented merits solving above problem. 
For over-determined models TLS estimates are given by 



argmin llDllj,, s.t.{B + D) 

D.x 



0, 



where || • |||, represents Forbenius-form operator. With the 
assumption of vec{D) ^ N{0,a'^), (22] gives the equivalent 
solutions as 

Axr 



arg mm ■ 



1 + 11x11 



(2) 



The distinct objective of the present paper is twofold: 
developing efficient solvers for fully-perturbed linear models, 
and accounting for sparsity of x. To achieve these goals, 
following optimization problem must be sovled 



x = arg min [Pill, +7||x||^] 



(3) 



where 7 > 0, and < p < 1. In the £p-teim forces the 
quadratic sum of perturbations to be minimal while the ^p-term 
forces sparsity of recovery |9|, |19|, and 7 controls tradeoff 
between above two terms. Developing efficient algorithms to 
get the local even global optimum of Q is the main goal. In 
next section, we will explain how to get the objective function 
and estimate the value of 7 with a bayesian formation, then 
develop the new method of optimization. 



This section develops an extension of FOCUSS, TLS- 
FOCUSS, to solve ([T]l using Bayesian framework [91 and 
main idea of TLS. For simplifying formulas, we assume 
a = ai = (7-2, that is vec(L)) iV(0, cr^). At the end of 
the section, we will introduce how to process the situation 

with (Tl (72- 

A. Bayesian Formulation 
From ([U, we obtain 

y-Ax = G(x)v, (4) 

where G{x) ~ [l,x-^] ® Imxm, v = vec(Z?), ((g) represents 
Kronecker product). Under Bayesian viewpoint, unknown vec- 
tor X is assumed to be random and independent of D. Then 
the MAP estimation of x can be obtained as: 



Xmap = argmaxlnp(x|y) 

X 

= argmax[lnp(y|x) 



Inp(x)] 



(5) 



This formula is general and offers considerable flexibility. In 
order to obtain optimality of the resultant estimates, another 
assumption must be made on the distributions of the solution 
vector X. As discussed in |15J, the elements of sparse x are 
assumed to be distributed as general Gaussian and indepen- 
dent, 

p(x) = C2exp|^-^f]|x[fc]|^^ , (6) 



where C2 is constant, < p < 1 and /3 is constant depended 

( where r(-) means Gamma 



- o-^ r(i/p) 



on p with P = 2 

function). Only one parameter characterizes the distribution in 
(|6]l. The pdf moves toward a uniform distribution as p 00 
and toward a very peaky distribution as p ^ 0. 

With v-iV(0, 0-2/) and G(x)G^(x) = (l + ||x||2)/, we have 

1 {y ^ Ax)" {y ^ Ax) 



lnp(y|x) 



(7) 



2ct2 1 + ||x||2 

where Ci is constant. With the densities of the perturbation 
vector V and the solution vector x, we can now proceed to find 
the MAP estimate as 



Xmap = arg mm 



^x 1 1 2 II IIP 
||„||2 + 7l|X||p 



(8) 



where 7 = cr^/P^- 

B. Derivation of TLS-FOCUSS 

The optimization problem (O is equivalent to 



where J(z') = 



argminz' J(z') 



f +7l|z'||P 



with 



z = 



B=[-y,Al 



(9) 



(10) 



To simplify the objective function,we normalize z' and get the 
equivalent form as 



\Bz\\ 



-7l|z|| 



s.L ||z||^ = L 



(11) 
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Using Lagrange multiplier method, the objective function can 
be rewritten as 



r(z) = ||i3z||2+7||z||^ + A(l-z^z) 



(12) 



where A is the Lagrange multiplier. The factored gradient 
approach developed in [23 1, an iterative method can be derived 
to minimize r(z. A necessary condition for the optimum 
solution z* is that it must satisfy WzT{z^,) = 0. We can get 



(B^B + an(z,))z, = Az*, 



(13) 



where 



Q!=P7/2, n(z) = diag \z[i 



|P-2 



J i=l,--- ,n+l^ 

So the iterative relaxation scheme can be constructed as 



(B^B + an(zfc_i))zfc = Azfc. 



(14) 



It is easily seen that A should be the minimal eigenvalue of 
objective matrix B + ali{ik-i). However, it's very hard 
to find it for two reasons: firstly, the minimal eigenvalue is 
likely close to zero because objective matrix is approximately 
singular; secondly, the dimension of matrix above is tremen- 
dous for most large scale application, which leads to a big 
computational burden for matrix inversion. (fT4] i implies that 



{B" B + a\l{zk-i)) 'zfc-izfe 



A 



From ( fTsT i, finding the minimal eigenvalue is taken place of 
by finding the maximal eigenvalue. The latter become much 
more well-posed. Moreover, with the aid of matrix inversion 
formula, we have 



{B" B + an{zk-i)Y^ 
= 1 {W^ - WlB" {al - BWIB^Y'bwI) , 
where = n-i(zfc_i). Let 

$fc - - WlB"{aI - BWlB")-^BWl 



then we obtain 



^feZfe = -Zfe. 



(16) 



(17) 



(18) 



It should be mentioned that the dimension of matrix 
al - BW^B^ is much less than that of matrix B^ B + 
an(zfe_i), so the cost of matrix inversion is extremely re- 
duced. Besides, we need only calculate the maximal eigen- 
value and corresponding eigenvector instead of all the eigen- 
value and eigenvector of ^k- That is to say, some highly 
efficient solver, such as Lanczos iteration, could be utilized 
to make the problem further simplified. 

Noting that the optimal problem dHJ is not global convex, 
the TLS-FOCUSS algorithm guarantees convergence to a local 
optimum. Once the initial point Zq is close to the true point, 
estimation of true value can be found through iterations. In 
this paper, we set Xo ~ {AA^)^^y, then Zq is set through 
substituting Xq into ( fTOl i and normalization of Zq. 

When the convergent solution z, is obtained, we can get 



Ti-TLS-FOCUSS = [^*2, ' ' ' 



/^*1 



(19) 



Algorithm 1 is the algorithmic description of TLS- 
FOCUSS. 
Algorithm 1 (TLS-FOCUSS): 
Input: Zq, B, a, p. 

1 Set Wk = diag f [|zfe_i[z]|'-?l ) (, and p e 

(0,1]); 

2 Calculate $fe = - WlB"{aI + BWlB^y^BWl- 

3 Compute the largest eigenvalue Afe and corresponding 
eigenvector Ufe of $fe using Lanczos method. 

4 Set Zfc = Ufe. 

5 If ||zfc - Zk-i\\l/\\zk-i\\l < e, exit; else goto step 1. 

C. Convergence and Sparsity 

To show that TLS-FOCUSS algorithm can approximately 
solve the sparse problem of ([T]i through iterative method, 
two key results should be obtained: i) TLS-FOCUSS is a 
convergent algorithm that it indeed reduces ,/(z) at each iterate 
step; ii) the convergence points of TLS-FOCUSS are sparse. 
proof of convergence: From ( fT4b we have 



BwBwqk + aqk - \W^qk = 0, 



(20) 



where Bw — BWk, 
an optimal solution: 



WY^k- And Qfe can be treated as 



(15) q, = argmin [\\Bwq\f + «|lq|l' + A(l - q"W^q)] . (21) 



From (|2TI) and the equivalence of optimization between Q 
and (fT2l i. z^ can be expressed a solution to an optimization 
problem: 



Zfe =argmin(3fe(z) 



where Qfe(z) = 



\Bz\\ 



+ a\\wrU^ 



(22) 



So TLS-FOCUSS algorithm can be considered to be a method 
of re-weighted ^2 -form minimization 15], ifTSl . Since z^ is the 
local unique solution to minimize Qk{z), we have 



Qfe(zfe) < (3fe(zfc_i) 



(23) 



with Zfc, Zfe_i located in the same small domain and z^ 7^ 

Zfe-l- 

And we can get the conclusion ifTSl that 



E 



{\Z2\IW~\Z1\IW) 



Zl[l\ 



=1 [z^n(zi)z2-zfn(zi)zi] 



(24) 



where n(z) = diag(|z[i]|P ^). With Zk-i and Zk (zk 7^ Zk^i) 
obtained from the (fc- l)th and fcth iteration of TLS-FOCUSS, 
we have 



J(Zfe) - J(Zfc_i) 



< 



k\\2 

Qki^k) - Qfe(Zfc-l) < 0, 



\Bz 



k-l\\2 



\\Zk-l\\2 



i_,W^'zk-i 
(25) 
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where and Zfc_i are obtained from the k-th and (fc — l)-th 
iteration step of TLS-FOCUSS. The first inequality follows 
from ( |24] | and the last inequality from ( |23] ). So the value of 
J(zfc) decreases as k increases. From dZST l and J(zfc) > 0, 
it can be concluded that TLS-FOCUSS is a convergent algo- 
rithm. ■ 
proof of sparsity: Assuming Zq is a local minima of 
J(z), Zq is also a local minima to an optimization problem: 
min^ \z[i]Y' s.t. [B + D)z — 0, which can be rewritten as 



mm > \x\i\ 

Y f ^ 



S.t. y = + E)^. + e. 



(26) 



Similarly shown in lH, ifTSll . ||24ll (especially p = 1), as 
an equivalence of fp-norm optimization above optimization 
problem can obtain the local minima which are necessary 
sparse. The provement of equivalence between ^o-norm and 
£p-norm about fully-perturbed model is aslo an open problem. 

Let z* be an fixed point of the algorithm, and therefore a 
solution of ( fTSl l. If z* is not sparse, it is not a local minima 
of ( l26l l, so there must be other points close to z* which can 
reduce J(z)f23l. Thus it can be concluded that only sparse 
solutions ai^e stable points of TLS-FOCUSS algorithm. ■ 

D. Robust Modification 

Note that we assumed the components of perturbation ma- 
trix [e, E] are i.i.d. (independent and identically distributed). 
Actually, only noise existing in the same channel is assumed to 
be i.i.d.. When e and E have the different distributed variances, 
it is necessary to normalize variances of perturbations before 
signal reconstruction. Assume that e and E are independent, 
and e - N{0,a'^li), \ec{E) - 7V(0,cr|/2). Then we have 
y — — G{x)\ with 



G(x) 



1, — x-' 



e 

^vec(£;). 



It can be seen v ^ N{0, a^I). For (|9]l, instead of ( fTOb we have 

z' = [i,^xT, B = [-y,-A]. 

Now TLS-FOCUSS algorithm can be used to recover the 
sparse signal. 

IV. SD-FOCUSS Algorithm 

TLS-FOCUSS needs to compute the maximal eigenvalue 
and its corresponding eigenvector of matrix $fc in every 
iteration. By utilizing Lanczos algorithm, TLS-FOCUSS al- 
gorithm can be speeded up greatly. However, it is still pos- 
sible to release much more the computation burden while 
the performance descends a little. In this section, a subopti- 
mal algorithm, named SD-FOCUSS (Synchronous Descending 
FOCUSS), is divised. 

Based on TLS model ([T]), Zhu in |FT9l| devised a sparse re- 
covery algorithm S-TLS. To optimize the objective function, S- 
TLS adopted iterative block coordinate descent method, yield- 
ing successive estimates of x with E fixed and alternately of 
E with X fixed until obtaining stable solutions. The algorithm 
needs several convergent procedures before final convergence. 



Different from S-TLS, SD-FOCUSS is more efficient, which 
only needs one convergent procedure, with estimating x and 
E synchronously in each iteration; meanwhile, SD-FOCUSS 
has lower computation complexity. 

A. Bayesian Formulation 

In this section, x and in ([T]| are both considered variants 
to be optimized. Assume that e ^ 7V(0, cr^/i), vec(i?) ^ 
Af(0,o'|/2), and e, E are independent. So we have 

e^e 



vec(i;)^vec(£;) ^ f \\E^^^ 

Pe{E) = Ci exp — ) = Gxp 



2al 



2al 



(27) 



Where C\, C2 are constant. The Bayesian formulation is 
described as 

[xmap,£^map] = argmaxlnp(x,i;|y) 

x.E 

= arg max [lnp(y |x, E) + Inp(x) + h\p{E)] . (28) 
Here we have 

lnp(y|x, E)^\\\y-{A + E) x|l^ + InCg. (29) 



B. Derivation of SD-FOCUSS 

From (|6]l ( |28] ) and ( |29] l, the objective function can be written 

as 

2 

J(x, E)^\\y-{A + EMl + %ir{E"E) + 7||x||^ (30) 

where tr(-) means trace of matrix and Xx{E^ E) = WEW^. The 
necessary condition of the optimal solution satisfies that partial 
differentiation to each component for J(x, E) is equal to zero, 
that is: 

a) VeJ{x,E^) = 0. 

We can get 

Vb J(x, E) = Exx" - (y - Ax)x" + aja^^E. 
So we can get the estimate of as a function of x: 

(y - ylx)x^ 



E,{x) 



(31) 



Here the fact of {XI + F"F)-^F" = F" {XI + FF")-^ is 
used. 

b) v,j(x*,£;) = 0. 

Referring to ifTSl . we can get the iterative relaxation scheme 
of X as 

Xk = WkA^{AkA^ + al)-^y, (32) 

where a ^ ^, Wk ^ diag [|xfc-i [i] .^-^ and 
Ak ^ {A + _E(xfe_i)) Wfc. There exists error inevitably when 
we estimate E, thus accuracy of estimating x will be affected. 
It is a suboptimal algorithm. 

Algorithm 2 is the algorithmic description of SD-FOCUSS. 

Algorithm 2 (SD-FOCUSS): 
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Input: y, Xq, Eq A, a, a2, p. 



normlized result recovered by regularized FOCUSS 



1 Set Wk = diag 

(0,1]); 

2 Calculate 



(, and p G 



(y- Axfc_i)xf_i 



and Ak = {A + Ek)Wk 



3 Calculate Xfe = WkA^{AkA^ + a/)" V; 

4 If llxfc - Xfc-i||l/||xfe_i||2 < e, exit; else goto step 1. 

C. Proof of Convergence 

Formula ( |32] | can be seen as Xfe — Wkbk, where bfe can be 
treated as an optimal solution, that is 

bfe = argmin||y-AfcW^feb|l^ + a||b|r 



(33) 



Alternately and equivalently, x^ can be expressed a solution 
to an optimization problem: 

Xfc = argmin(5fc(x), 

where Qk{x) = \\y - Akx\\l + a\\W-^x\\l (34) 

Referring to we can conclude that SD-FOCUSS is 

also a convergent algorithm. 



D. SD-FOCUSS Extension: MMV case 

Besides low computation, the breakthrough advantage of 
SD-FOCUSS is that it can be used in multiple measurement 
vectors (MMV) model, while TLS-FOCUSS and S-TLS ||T9] 
cannot fit this model or remain to be developed. Supposed 
yC) = {A + E)x^^^+e^^\ with / 1, - •• ,L, where yW e i?™ 
and x^'^ e i?". Suppose that the vectors x^''\l ~ I,-- - ,L 
are sparse and have the same sparsity profile, and let Y ~ 
[yW,...,yW],X = [xW,...,xW]. 

The objective function for MMV case is expressed as 



J{X,E)^\\Y -{A + E)X\\'p- 



p/2 



:^\\ErF 



7E E 



(35) 



The weight matrix Wk can be re-expressed as ||6l 

Wk =diag (cfe with Ck[i] = ( E (^fe-i W) 

\i=i 

Then formula ( |32] i can be rewritten as 

Xk^WkA^iAkA^ + aI)-'Y 
For WeJ{x,E) = we can renew dJTT i as 

^2 




Ek 



{Y - AXk-i) —I + X^_^Xk-i 



X 



H 

k-1 



(38) 



Then the Algorithm 2 can be modified to fit MMV model 
as Algorithm 3. 

Algorithm 3 (MMV SD-FOCUSS): 
Input: y, Xq, Eq A, a, a2, p. 



15 20 25 30 



(a) Result recovered by Regulaiized FOCUSS: weak signal is loss 
normlized result recovered by TLS-FOCUSS 



(b) Result of recovered by TLS-FOCUSS: weak signal is found 
Fig. 1. Result of weak signal recovery with m = 20, n = 30 



1 Set Wk=dmg[[ck[i]^-^^^^ „ 



where Ck[i] = [T,i=i 
2 Calculate 



1/2 



pe (0,1]); 



Ek = {Y- AXk^i) [a'a^'l + ^f-i^fe-i] ' X^_, 

and Ak = {A + Ek)Wk; 

3 Calculate Xk = WkA^ {AkA^ + aI)-'^Y; 

4 If \\Xk - Xk-i\\ll\\Xk-i\\l < e, exit; else goto step 1. 



V. Simulation Results 

The parameters in this paper are set as: norm-factor p = 0.5, 
convergence threshold e = 0.01. In each Monte Carlo simu- 
lation, 1000 trials are carried out independently. In each trial, 
the mxn dictionary A is chosen as Gaussian random matrix, 
entries of which are independently, identically and normally 
distributed. In order to analyze the mentioned algorithms, the 
true sparse solution has to be known, and it is hard to know 
in practice problems. 

The algorithm in one simulation is considered to be success- 
ful if all nonzero-locations of x are found exactly; otherwise, 
the algorithm is considered to be failed. 
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- * - 


FOCUSS 


- V - 


Regularized FOCUSS 




TLS-FOCUSS 


-□- 


SD-FOCUSS 


■■<1 


S-TLS 



14 16 
SNR/dB 
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+ - FOCUSS 

TLS-FOCUSS 
-□- ■ SD-FOCUSS 
S-TLS 



0.1 0.15 0.2 0.25 

Fraclion of sparsily entries [k/n] 



(a) success probability of algorithms in finding the support set con'ectly 



(a) percentage success with randiness distribution in sparse entries 



- * - 


FOCUSS 


- ? - 


Regularized FOCUSS 




TLS-FOCUSS 


-□- 


SD-FOCUSS 




S-TLS 




14 16 
SNR/dB 



(b) RMSE of signal ampHtude recovery 
Fig. 2. Performance of involved algorithms with m = 20, n = 30 



- + - FOCUSS 
— e — TLS-FOCUSS 
-□- SD-FOCUSS 
S-TLS 




0.1 0.15 0.2 0.25 

Fraction of sparsily entries (k/n) 



(b) percentage success with the same amplitude in sparse entries 

Fig. 3. percentage success of involved algorithms with different k/m. m - 
20, n = 30 



A. Single Measurement Vector Case 

This subsection shows the advantages of recovering ability 
of new algorithms from TLS model with numerical simulation. 
Let X be a s-sparse vector, i.e. ||x||o ~ s, and let the average 
power of X be normalized, i.e. ^ - = 1. In each trial, 
entries of matrix [e, E] are also independently and identically 
Gaussian distributee^] with mean zero and variance a^. Then 
overall SNR can be represented as 1/cr^. The indices of 
nonzero coordinate set T are chosen randomly from a discrete 
uniform distribution U{1,N) (without repetition). 

In following simulations, besides TLS-FOCUSS and SD- 
FOCUSS, other algorithms will be involved: standard FO- 
CUSS S, Regularized FOCUSS |[15l, and S-TLS lfT9l . 

In Fig. [T] and Fig. |2] the number of rows and columns 
of dictionary matrix are set to 20 and 30 respectively. In 
Fig. [1] SNR is set to 15 dB, T = [3, 15, 25] and xt = 
[0.4139, -0.9186, -1.4819]^. It can be seen from Fig. [T]that 
TLS-FOCUSS does much better than FOCUSS in extract- 
ing weak signal when dictionary and measurement are both 
corrupted. For TLS-FOCUSS, the position and ampUtude of 

'if the variances of generalizing e and E are different, the performance of 
TLS-FOCUSS will not change, while the performances of the other algorithms 
will be affected. 



signal are both recovered excellently; the result of FOCUSS 
is failed, for weak signal is buried in "False Peak" brought 
by perturbation on dictionary and can not be distinguished 
correctly. Fig. |2A) shows the statistical results of percentage 
success, and Fig. EJB) shows the statistical root-mean-square 
error (RMSE) of signal amplitude recovery when algorithms 
can find the nonzero-coordinate T correctly under different 
SNR scenes. TLS-FOCUSS and SD-FOCUSS are presented 
to be more robust from Fig. |2(a) and perform much better on 



amplitude recovery from Fig. 2( 



Fig. |3] shows the percentage-success curves of algorithms 
with different k/m. In the simulation, m = 20, n ~ 30, k = 
1,2, - •• ,10, SNR=15dB and entries of x^ are set to obey 
i.i.d. normal distribution in Fig. |3(a)| and 1 in Fig. |3(b)| It can 
be seen from Fig. |3] that, TLS-FOCUSS and SD-FOCUSS 
perform always better than common algorithms (FOCUSS) 
and S-TLS designed to solve fully-perturbed model as k/m 
changes. 

In the simulations of Fig.|4]and Table|I] m = 128, n = 512, 
s = 3 and xt = (1, —1, l)^/\/3- With smooth curves. Fig. |4] 
shows that the recovery performance of TLS-FOCUSS in 
this scenario is much better than the other algorithms; SD- 
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0.22 
0.2 

o.iaf- 

0.16 
0.14 
0.12 
0.1 
0.08 
0.06 
0.04 



0.02 



- * - 


FOCUSS 


- 9 - 


Regularized FOCUSS 




TLS-FOCUSS 


-□- 


SD-FOCUSS 




S-TLS 




SNR 


FOCUSS 


RegFOC 


TLS-FOC 


SD-FOC 


S-TLS 


(dB) 


(sec) 


(sec) 


(sec) 


(sec) 


(sec) 


10 


0.1284 


0.0160 


0.5841 


0.3008 


5.1528 


11 


0.1298 


0.0182 


0.6530 


0.3513 


5.3670 


12 


0.1274 


0.0185 


0.6291 


0.3276 


5.3779 


13 


0.1218 


0.0158 


0.5852 


0.3010 


5.2276 


14 


0.1215 


0.0156 


0.6001 


0.2964 


5.2652 


15 


0.1211 


0.0156 


0.5863 


0.2959 


5.3563 


16 


0.1202 


0.0155 


0.5858 


0.2961 


5.3963 


17 


0.1213 


0.0156 


0.5867 


0.2958 


5.3639 


18 


0.1211 


0.0155 


0.5880 


0.2966 


5.3069 


19 


0.1213 


0.0163 


0.6104 


0.3121 


5.2285 


20 


0.1221 


0.0156 


0.5921 


0.2965 


5.1877 



TABLE I 

Run-time of algorithms with m = 128, n = 512. The simulations 

ARE DONE IN MATLAB 7.8 ON A CORE 2, 3.0-GHZ, 2-GBYTE RAM PC 
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12 14 16 

SNR/dB 



Fig. 4. RMSE of signals recovery in the condition of m = 128, n = 512. 



FOCUSS is superior to S-TLS in low SNR, and inferior to 
S-TLS in high SNR. Table U shows run-times of mentioned 
algorithms under the same condition. In order to obtain a 
measure of the computational complexity, the average CPU 
times for each algorithm consumeing is tabulated in Table U 
It can be seen that, as the same classified algorithms TLS- 
FOCUSS and SD-FOCUSS ai-e much faster than S-TLS. 

By comparison with other algorithms, it can be concluded 
that TLS-FOCUSS and SD-FOCUSS have the complete ad- 
vances in percentage succuss, accurate reconstruction and 
computational speed. And TLS-FOCUSS has the higher suc- 
cess percentage and more accurate reconstruction than SD- 
FOCUSS while SD-FOCUSS is faster than TLS-FOCUSS. 

B. MMV Case 

In this simulation we consider the performance of SD- 
FOCUSS in MMV case. X is a sparse matrix with L columns 
and only s rows with nonzero entries. In each trial, the indices 
of nonzero rows in X are chosen randomly from a discrete 
uniform distribution, and the amplitudes of the row entries 
are generalized randomly from a standard normal distribution; 
entries of both E and [e*^'-'];=i^... ,l are independently Gaussian 
distributed with mean zero and variance a^. The overall SNR 
is 1/(7^. The measurement matrix can expressed as 



Y = {A + E)X 



The relative MSB between the true and estimate solution is 
defined as [|6j| 



MSB 



\X 



\x\\l 



In following simulations, besides SD-FOCUSS, the other 
algorithms will be involved, containing: MMV FOCUSS IS), 
Regularized MMV FOCUSS |6|, and MMV OMP |6|. 

The number of rows and columns of dictionary A are set 
to 20 and 30 respectively, and let s = 7. Two quantities are 
varied in this experiment: SNR and L. Fig. |5] and Fig. |6] show 
success-probability curves and MSB curves respectively when 
i = 2, 5, 6. It can be found that as L becomes larger, success 



numbers become larger; however, MSB curves seem to be 
unchanged for it is related with perturbation and unrelated with 
L. MMV SD-FOCUSS performs better than other algorithms. 

VI. Conclusion 

In this paper, through extending FOCUSS algorithms, we 
have proposed two new algorithms, TLS-FOCUSS and SD- 
FOCUSS, to recover the sparse vector from an underdeter- 
mined system when the measurements and dictionary ma- 
trix are both perturbed. The convergence of algorithms was 
considered. Then we appHed SD-FOCUSS in MMV model 
with a row-sparsity structure. The simulations showed our 
approaches performed better than other present algorithms 
in computational complexity, percentage success and RMSB 
of signal amplitude recovery. The benefits of TLS-FOCUSS 
and SD-FOCUSS make them be good candidates of sparse 
recovery algorithms for more practical apphcations. 
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